Interactivity model for shared feedback on mobile devices

ABSTRACT

A system that produces a dynamic haptic effect and generates a drive signal that includes a gesture signal and a real or virtual device sensor signal. The haptic effect is modified dynamically based on both the gesture signal and the real or virtual device sensor signal such as from an accelerometer or gyroscope, or by a signal created from processing data such as still images, video or sound. The haptic effect may optionally be modified dynamically by using the gesture signal and the real or virtual device sensor signal and a physical model, or may optionally be applied concurrently to multiple devices which are connected via a communication link. The haptic effect may optionally be encoded into a data file on a first device. The data file is then communicated to a second device and the haptic effect is read from the data file and applied to the second device.

FIELD OF THE INVENTION

One embodiment is directed generally to a user interface for a device,and in particular to producing a dynamic haptic effect using multiplegesture signals and real or virtual device sensor signals.

BACKGROUND INFORMATION

Electronic device manufacturers strive to produce a rich interface forusers. Conventional devices use visual and auditory cues to providefeedback to a user. In some interface devices, kinesthetic feedback(such as active and resistive force feedback) and/or tactile feedback(such as vibration, texture, and heat) is also provided to the user,more generally known collectively as “haptic feedback” or “hapticeffects”. Haptic feedback can provide cues that enhance and simplify theuser interface. Specifically, vibration effects, or vibrotactile hapticeffects, may be useful in providing cues to users of electronic devicesto alert the user to specific events, or provide realistic feedback tocreate greater sensory immersion within a simulated or virtualenvironment.

In order to generate vibration effects, many devices utilize some typeof actuator. Known actuators used for this purpose include anelectromagnetic actuator such as an Eccentric Rotating Mass (“ERM”) inwhich an eccentric mass is moved by a motor, a Linear Resonant Actuator(“LRA”) in which a mass attached to a spring is driven back and forth,or a “smart material” such as piezoelectric, electro-active polymers orshape memory alloys.

Traditional architectures provide haptic feedback only when triggeredeffects are available, and must be carefully designed to make sure thetiming of the haptic feedback is correlated to user initiated gesturesor system animations. However, because these user gestures and systemanimations have variable timing, the correlation to haptic feedback maybe static and inconsistent and therefore less compelling to the user.Further, device sensor information is typically not used in combinationwith gestures to produce haptic feedback.

Therefore, there is a need for an improved system of providing a dynamichaptic effect that includes multiple gesture signals and device sensorsignals. There is a further need for providing concurrent hapticfeedback to multiple devices which are connected via a communicationlink.

SUMMARY OF THE INVENTION

One embodiment is a system that produces a dynamic haptic effect andgenerates a drive signal that includes a gesture signal and a real orvirtual device sensor signal. The haptic effect is modified dynamicallybased on both the gesture signal and the real or virtual device sensorsignal such as from an accelerometer or gyroscope, or by a signalcreated from processing data such as still images, video or sound. Thehaptic effect may optionally be modified dynamically by using thegesture signal and the real or virtual device sensor signal and aphysical model. The haptic effect may optionally be applied concurrentlyto multiple devices which are connected via a communication link. Thehaptic effect may optionally be encoded into a data file on a firstdevice. The data file is then communicated to a second device and thehaptic effect is read from the data file and applied to the seconddevice.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a haptically-enabled system according toone embodiment of the present invention.

FIG. 2 is a cut-away perspective view of an LRA implementation of ahaptic actuator according to one embodiment of the present invention.

FIG. 3 is a cut-away perspective view of an ERM implementation of ahaptic actuator according to one embodiment of the present invention.

FIGS. 4A-4C are views of a piezoelectric implementation of a hapticactuator according to one embodiment of the present invention.

FIGS. 5A-5C are screen views of a user initiated dynamic haptic effectaccording to one embodiment of the present invention.

FIGS. 6A-6F are screen views of encoding a haptic effect into a datafile according to one embodiment of the present invention.

FIG. 7 is a screen view of a user initiated dynamic haptic effectaccording to one embodiment of the present invention.

FIGS. 8A-8E are screen views of applying a haptic effect concurrently tomultiple devices according to one embodiment of the present invention.

FIG. 9 is a flow diagram for producing a dynamic haptic effect with agesture signal and a device sensor signal according to one embodiment ofthe present invention.

FIG. 10 is a flow diagram for concurrently applying a haptic effect tomultiple devices according to one embodiment of the present invention.

FIG. 11 is a flow diagram for encoding and applying a haptic effectusing a data file according to one embodiment of the present invention.

DETAILED DESCRIPTION

As described below, a dynamic effect refers to a haptic effect thatevolves over time as it responds to one or more input parameters. Adynamic effect signal can be any type of signal, but does notnecessarily have to be complex. For example, a dynamic effect signal maybe a simple sine wave that has some property such as phase, frequency,or amplitude that is changing over time or reacting in real timeaccording to a mapping schema which maps an input parameter onto achanging property of the effect signal. An input parameter may be anytype of input capable of being provided by a device, and typically maybe any type of signal such as a device sensor signal. A device sensorsignal may be generated by any means, and typically may be generated bycapturing a user gesture with a device. Dynamic effects may be veryuseful for gesture interfaces, but the use of gestures or sensors arenot necessarily required to create a dynamic signal.

One common scenario that does not involve gestures directly is definingthe dynamic haptic behavior of an animated widget. For example, when auser scrolls a list, it is not typically the gesture that is subjectedto haptification but instead the motion of the widget in response to thegesture that will feel most intuitive when haptified. In the scroll listexample, gently sliding a virtual scroll bar may generate a dynamichaptic feedback that changes according to the speed of the scrolling,but flinging the scroll bar may produce dynamic haptics even after thegesture has ended. This creates the illusion that the widget has somephysical properties and it provides the user with information about thestate of the widget such as speed or whether it is in motion.

A gesture is any movement of the body that conveys meaning or userintent. It will be recognized that simple gestures may be combined toform more complex gestures. For example, bringing a finger into contactwith a touch sensitive surface may be referred to as a “finger on”gesture, while removing a finger from a touch sensitive surface may bereferred to as a separate “finger off” gesture. If the time between the“finger on” and “finger off” gestures is relatively short, the combinedgesture may be referred to as “tapping”; if the time between the “fingeron” and “finger off” gestures is relatively long, the combined gesturemay be referred to as “long tapping”; if the distance between the twodimensional (x,y) positions of the “finger on” and “finger off” gesturesis relatively large, the combined gesture may be referred to as“swiping”; if the distance between the two dimensional (x,y) positionsof the “finger on” and “finger off” gestures is relatively small, thecombined gesture may be referred to as “smearing”, “smudging” or“flicking”. Any number of two dimensional or three dimensional simple orcomplex gestures may be combined in any manner to form any number ofother gestures, including, but not limited to, multiple finger contacts,palm or first contact, or proximity to the device.

FIG. 1 is a block diagram of a haptically-enabled system 10 according toone embodiment of the present invention. System 10 includes a touchsensitive surface 11 or other type of user interface mounted within ahousing 15, and may include mechanical keys/buttons 13. Internal tosystem 10 is a haptic feedback system that generates vibrations onsystem 10. In one embodiment, the vibrations are generated on touchsurface 11.

The haptic feedback system includes a processor 12. Coupled to processor12 is a memory 20 and an actuator drive circuit 16, which is coupled toa haptic actuator 18. Processor 12 may be any type of general purposeprocessor, or could be a processor specifically designed to providehaptic effects, such as an application-specific integrated circuit(“ASIC”). Processor 12 may be the same processor that operates theentire system 10, or may be a separate processor. Processor 12 candecide what haptic effects are to be played and the order in which theeffects are played based on high level parameters. In general, the highlevel parameters that define a particular haptic effect includemagnitude, frequency and duration. Low level parameters such asstreaming motor commands could also be used to determine a particularhaptic effect. A haptic effect may be considered dynamic if it includessome variation of these parameters when the haptic effect is generatedor a variation of these parameters based on a user's interaction.

Processor 12 outputs the control signals to drive circuit 16 whichincludes electronic components and circuitry used to supply actuator 18with the required electrical current and voltage to cause the desiredhaptic effects. System 10 may include more than one actuator 18, andeach actuator may include a separate drive circuit 16, all coupled to acommon processor 12. Memory device 20 can be any type of storage deviceor computer-readable medium, such as random access memory (RAM) orread-only memory (ROM). Memory 20 stores instructions executed byprocessor 12. Among the instructions, memory 20 includes an actuatordrive module 22 which are instructions that, when executed by processor12, generate drive signals for actuator 18 while also determiningfeedback from actuator 18 and adjusting the drive signals accordingly.The functionality of module 22 is discussed in more detail below. Memory20 may also be located internal to processor 12, or any combination ofinternal and external memory.

Touch surface 11 recognizes touches, and may also recognize the positionand magnitude or pressure of touches on the surface. The datacorresponding to the touches is sent to processor 12, or anotherprocessor within system 10, and processor 12 interprets the touches andin response generates haptic effect signals. Touch surface 11 may sensetouches using any sensing technology, including capacitive sensing,resistive sensing, surface acoustic wave sensing, pressure sensing,optical sensing, etc. Touch surface 11 may sense multi-touch contactsand may be capable of distinguishing multiple touches that occur at thesame time. Touch surface 11 may be a touchscreen that generates anddisplays images for the user to interact with, such as keys, dials,etc., or may be a touchpad with minimal or no images.

System 10 may be a handheld device, such as a cellular telephone, PDA,computer tablet, gaming console, etc. or may be any other type of devicethat provides a user interface and includes a haptic effect system thatincludes one or more ERMs, LRAs, electrostatic or other types ofactuators. The user interface may be a touch sensitive surface, or canbe any other type of user interface such as a mouse, touchpad,mini-joystick, scroll wheel, trackball, game pads or game controllers,etc. In embodiments with more than one actuator, each actuator may havea different output capability in order to create a wide range of hapticeffects on the device. Each actuator may be any type of haptic actuatoror a single or multidimensional array of actuators.

FIG. 2 is a cut-away side view of an LRA implementation of actuator 18in accordance to one embodiment. LRA 18 includes a casing 25, amagnet/mass 27, a linear spring 26, and an electric coil 28. Magnet 27is mounted to casing 25 by spring 26. Coil 28 is mounted directly on thebottom of casing 25 underneath magnet 27. LRA 18 is typical of any knownLRA. In operation, when current flows thru coil 28 a magnetic fieldforms around coil 28 which in interaction with the magnetic field ofmagnet 27 pushes or pulls on magnet 27. One current flowdirection/polarity causes a push action and the other a pull action.Spring 26 controls the up and down movement of magnet 27 and has adeflected up position where it is compressed, a deflected down positionwhere it is expanded, and a neutral or zero-crossing position where itis neither compressed or deflected and which is equal to its restingstate when no current is being applied to coil 28 and there is nomovement/oscillation of magnet 27.

For LRA 18, a mechanical quality factor or “Q factor” can be measured.In general, the mechanical Q factor is a dimensionless parameter thatcompares a time constant for decay of an oscillating physical system'samplitude to its oscillation period. The mechanical Q factor issignificantly affected by mounting variations. The mechanical Q factorrepresents the ratio of the energy circulated between the mass andspring over the energy lost at every oscillation cycle. A low Q factormeans that a large portion of the energy stored in the mass and springis lost at every cycle. In general, a minimum Q factor occurs withsystem 10 is held firmly in a hand due to energy being absorbed by thetissues of the hand. The maximum Q factor generally occurs when system10 is pressed against a hard and heavy surface that reflects all of thevibration energy back into LRA 18.

In direct proportionality to the mechanical Q factor, the forces thatoccur between magnet/mass 27 and spring 26 at resonance are typically10-100 times larger than the force that coil 28 must produce to maintainthe oscillation. Consequently, the resonant frequency of LRA 18 ismostly defined by the mass of magnet 27 and the compliance of spring 26.However, when an LRA is mounted to a floating device (i.e., system 10held softly in a hand), the LRA resonant frequency shifts upsignificantly. Further, significant frequency shifts can occur due toexternal factors affecting the apparent mounting weight of LRA 18 insystem 10, such as a cell phone flipped open/closed or the phone heldtightly.

FIG. 3 is a cut-away perspective view of an ERM implementation ofactuator 18 according to one embodiment of the present invention. ERM 18includes a rotating mass 301 having an off-center weight 303 thatrotates about an axis of rotation 305. In operation, any type of motormay be coupled to ERM 18 to cause rotation in one or both directionsaround the axis of rotation 305 in response to the amount and polarityof voltage applied to the motor. It will be recognized that anapplication of voltage in the same direction of rotation will have anacceleration effect and cause the ERM 18 to increase its rotationalspeed, and that an application of voltage in the opposite direction ofrotation will have a braking effect and cause the ERM 18 to decrease oreven reverse its rotational speed.

One embodiment of the present invention provides haptic feedback bydetermining and modifying the angular speed of ERM 18. Angular speed isa scalar measure of rotation rate, and represents the magnitude of thevector quantity angular velocity. Angular speed or frequency ω, inradians per second, correlates to frequency v in cycles per second, alsocalled Hz, by a factor of 2π. The drive signal includes a drive periodwhere at least one drive pulse is applied to ERM 18, and a monitoringperiod where the back electromagnetic field (“EMF”) of the rotating mass301 is received and used to determine the angular speed of ERM 18. Inanother embodiment, the drive period and the monitoring period areconcurrent and the present invention dynamically determines the angularspeed of ERM 18 during both the drive and monitoring periods.

FIGS. 4A-4C are views of a piezoelectric implementation of a hapticactuator 18 according to one embodiment of the present invention. FIG.4A shows a disk piezoelectric actuator that includes an electrode 401, apiezo ceramics disk 403 and a metal disk 405. As shown in FIG. 4B, whena voltage is applied to electrode 401, the piezoelectric actuator bendsin response, going from a relaxed state 407 to a transformed state 409.When a voltage is applied, it is that bending of the actuator thatcreates the foundation of vibration. Alternatively, FIG. 4C shows a beampiezoelectric actuator that operates similarly to a disk piezoelectricactuator by going from a relaxed state 411 to a transformed state 413.

FIGS. 5A-5C are screen views of a user initiated dynamic haptic effectaccording to one embodiment of the present invention. Dynamic effectsinvolve changing a haptic effect provided by a haptic enabled device inreal time according to an interaction parameter. An interactionparameter can be derived from any two-dimensional or three-dimensionalgesture using information such as the position, direction and velocityof a gesture from a two-dimensional on-screen display such as on amobile phone or tablet computer, or a three-dimensional gesturedetection system such as a video motion capture system or an electronicglove worn by the user, or by any other 2D or 3D gesture input means.FIG. 5A shows a screen view of a mobile device having a touch sensitivedisplay which displays one photograph out of a group of photographs.FIG. 5B shows a screen view of a user gesture using a single indexfinger being swiped across the touch sensitive display from right toleft in order to display the next photograph. Multiple inputs from theindex finger are received from the single gesture. Each of the multipleinputs may occur at a different time and may indicate a different twodimensional position of the contact point of the index finger with thetouch sensitive display.

FIG. 5C shows a screen view of the next photograph being displayed inconjunction with a dynamic haptic effect. Based upon the one or moreinputs from the one or more user gestures in FIG. 5B, a dynamic hapticeffect is provided during the user gesture and continuously modified asdetermined by the interaction parameter. The dynamic haptic effect mayspeed up or slow down, increase or decrease in intensity, or change itspattern or duration, or change in any other way, in real-time accordingto such elements as the speed, direction, pressure, magnitude, orduration of the user gesture itself, or based on a changing property ofa virtual object such as the number of times an image has been viewed.The dynamic haptic effect may further continue and may further bemodified by the interaction parameter even after the user gesture hasstopped. For example, in one embodiment the dynamic haptic effect may bestop immediately at the end of the user gesture, or in anotherembodiment the dynamic haptic effect may optionally fade slowly afterthe end of the user gesture according to the interaction parameter. Theeffect of providing or modifying a dynamic haptic effect in real-timeduring and even after a user gesture is that no two gestures such aspage turns or finger swipes will feel the same to the user. That is, thedynamic haptic effect will always be unique to the user gesture, therebycreating a greater sense connectedness to the device and a morecompelling user interface experience for the user as compared to asimple static haptic effect provided by a trigger event.

The interaction parameter may also be derived from device sensor datasuch as whole device acceleration, gyroscopic information or ambientinformation. Device sensor signals may be any type of sensor inputenabled by a device, such as from an accelerometer or gyroscope, or anytype of ambient sensor signal such as from a microphone, photometer,thermometer or altimeter, or any type of bio monitor such as skin orbody temperature, blood pressure (BP), heart rate monitor (HRM),electroencephalograph (EEG), or galvanic skin response (GSR), orinformation or signals received from a remotely coupled device, or anyother type of signal or sensor including, but not limited to, theexamples listed in TABLE 1 below.

TABLE 1 LIST OF SENSORS For the purposes of physical interaction design,a sensor is a transducer that converts a form of energy into anelectrical signal, or any signal that represents virtual sensorinformation. Acceleration Accelerometer Biosignals Electrocardiogram(ECG) Electroencephalogram (EEG) Electromyography (EMG)Electrooculography (EOG) Electropalatography (EPG) Galvanic SkinResponse (GSR) Distance Capacitive Hall Effect Infrared Ultrasound FlowUltrasound Force/pressure/strain/bend Air Pressure Fibre Optic SensorsFlexion Force-sensitive Resistor (FSR) Load Cell LuSense CPS² 155Miniature Pressure Transducer Piezoelectric Ceramic & Film Strain GageHumidity Hygrometer Linear position Hall Effect Linear Position (Touch)Linear Potentiometer (Slider) Linear Variable Differential Transformer(LVDT) LuSense CPS² 155 Orientation/inclination Accelerometer Compass(Magnetoresistive) Inclinometer Radio Frequency Radio FrequencyIdentification (RFID) Rotary position Rotary Encoder RotaryPotentiometer Rotary velocity Gyroscope Switches On-Off SwitchTemperature Temperature Vibration Piezoelectric Ceramic & Film Visiblelight intensity Fibre Optic Sensors Light-Dependent Resistor (LDR)

Active or ambient device sensor data may be used to modify the hapticfeedback based any number of factors relating to a user's environment oractivity. For example, an accelerometer device sensor signal mayindicate that a user is engaging in physical activity such as walking orrunning, so the pattern and duration of the haptic feedback should bemodified to be more noticeable to the user. In another example, amicrophone sensor signal may indicate that a user is in a noisyenvironment, so the amplitude or intensity of the haptic feedback shouldbe increased. Sensor data may also include virtual sensor data which isrepresented by information or signals that are created from processingdata such as still images, video or sound. For example, a video gamethat has a virtual racing car may dynamically change a haptic effectbased the car velocity, how close the car is to the camera viewingangle, the size of the car, and so on.

The interaction parameter may optionally incorporate a mathematicalmodel related to a real-world physical effect such as gravity,acceleration, friction or inertia. For example, the motion andinteraction that a user has with an object such as a virtual rollingball may appear to follow the same laws of physics in the virtualenvironment as an equivalent rolling ball would follow in a non-virtualenvironment.

The interaction parameter may optionally incorporate an animation indexto correlate the haptic output of a device to an animation or a visualor audio script. For example, an animation or script may play inresponse to a user or system initiated action such as opening orchanging the size of a virtual window, turning a page or scrollingthrough a list of data entries.

Two or more gesture signals, device sensor signals or physical modelinputs may be used alone or in any combination with each other to createan interaction parameter having a difference vector. A difference vectormay be created from two or more scalar or vector inputs by comparing thescalar or vector inputs with each other, determining what change ordifference exists between the inputs, and then generating a differencevector which incorporates a position location, direction and magnitude.Gesture signals may be used alone to create a gesture difference vector,or device sensor signals may be used alone to create a device signaldifference vector.

FIGS. 6A-6F are screen views of encoding a haptic effect into a datafile according to one embodiment of the present invention. In order tofacilitate dynamic haptic feedback between two or more users, it is notnecessary to have low latency or pseudo synchronous communication of thehaptic effect. Instead, one embodiment of the present invention enablesremote haptic interaction that takes place out of real time by encodinghaptic effect data into a shared data file. An example of such a nonreal time interaction is encoding the haptic effect taken from a digitaldrawing surface. FIG. 6A shows a default screen view of a virtual“frost” application running on a handheld or mobile device having adigital drawing surface and a haptic actuator. FIG. 6B shows the screenview of a “frosted” screen, created from the default screen view inresponse to user gestures or device sensor signals such as blowing intoa microphone on the handheld device. Once the screen is frosted, FIG. 6Cshows the creation of a stylized face pattern drawn in the frostaccording to gestures provided by the first user. The frosted screen andstylized face are stored in a data file in a format that supports eitherraster or vector depiction of images, and optionally any other data ormetadata necessary for subsequent reproduction of the image such asinformation about stored gestures or device sensor information.

A haptic effect corresponding to the motions used to create the stylizedface is stored or encoded into the data file concurrently with the otherimage information in the data file. The haptic effect information may bestored in any way that permits the reproduction of the haptic effectalong with the image. The data file is then communicated to a seconddevice having a haptic actuator via any file transfer mechanism orcommunication link. FIG. 6D shows the second device reading the storedgesture or device sensor signal from the data file on the second deviceand displaying the default frosted screen view. FIG. 6E shows how thestylized face is then subsequently displayed on the second device. Adrive signal is also applied to the haptic actuator on the second deviceaccording to the gesture or device sensor signal stored in the file.

The second user may optionally collaborate with the first user to createa combined data file by providing additional gestures or device sensorsignals to add the virtual message “Hi” on the drawing, along with anycorresponding haptic effect generated from the virtual message andstored in the data file. FIG. 6F shows the final collaborative screenview which combines gestures and device sensor signals from the firstand second users along with the corresponding haptic effect data.Gestures, device sensor signals and haptic effect data generated by bothusers are stored or encoded into the data file as a combinedcollaborative document which can subsequently be communicated betweenthe users or to other users for further input, modification orcollaboration. Although the above example describes a digital drawingsurface, it will be recognized that many other types of user gesturesand device sensor data may be stored or encoded with haptic effectsignals in any type of data file in virtually any format, withoutlimitation.

FIG. 7 is a screen view of a user initiated dynamic haptic effectaccording to one embodiment of the present invention. A filmstripapplication for displaying or selecting photographs is shown running atthe bottom of a handheld or mobile device having a touch sensitivesurface and a haptic actuator. By using gestures or device sensor data,a user may scroll the filmstrip from left to right or right to left, andthe filmstrip application may then dynamically provide a haptic effectfor a first photograph 701 which is different from a haptic effect for asecond photograph 703 based upon the gestures or device sensor data.Once the user has initiated the selection of a photograph through agesture, the system may provide an animation to visually show thefilmstrip in motion along with a corresponding haptic animationcomponent. Subsequent user gestures or device sensor informationreceived during the filmstrip animation may cause the haptic effect tochange along with any associated change in the animation. For example,if the filmstrip animation is moving too slow or too fast, the user mayspeed it up or slow it down in real time with a gesture and thecorresponding haptic effect component will also change dynamically inreal time along with the animation.

FIGS. 8A-8E are screen views of applying a haptic effect concurrently tomultiple devices according to one embodiment of the present invention.FIG. 8A shows a screen view of a haptic enabled handheld or mobiledevice of a first user 801, along with a visual thumbnail view of asecond user 803 also having a haptic enabled handheld or mobile device.The first and second devices may be connected in real time via any typeof communication link, including but not limited to electronic,cellular, wireless, wi-fi, optical, infrared, acoustic, Bluetooth, USB,Firewire, Thunderbolt or Ethernet.

FIG. 8B shows the first user selecting an application to sharephotographs between the two users. Upon selecting the application, FIG.8C shows the first photograph in the album, and FIG. 8D shows the firstuser applying a scrolling gesture to select the second photograph in thealbum by scrolling the photos from right to left. A corresponding hapticeffect is provided to the first user during the scrolling gesture.Because the first and second devices are connected in real time via thecommunication link, FIG. 8E shows the screen view of the second userwhich visually shows the same photograph as being displayed concurrentlyto the first user. Because of the real time link between the twodevices, the second user is able to concurrently view the same photos asthe first user. The second user also experiences in real time a similarhaptic effect for each gesture and photo as provided for the first user.In one embodiment, user gestures and haptic effects generated by thesecond user may be optionally communicated concurrently to the firstuser via the communication link, creating a real time bi-directionalhaptic link between the first and second devices. For example, the firstuser may scroll to the second photo, the second user may then scroll tothe third photo, and so on. It will be recognized that many other typesof user gestures, device sensor data and haptic effects may becommunicated between two or more devices in real time withoutlimitation.

FIG. 9 is a flow diagram for producing a dynamic haptic effect with agesture signal and a device sensor signal according to one embodiment ofthe present invention. In one embodiment, the functionality of the flowdiagram of FIG. 9 is implemented by software stored in memory or othercomputer readable or tangible medium, and executed by a processor. Inother embodiments, the functionality may be performed by hardware (e.g.,through the use of an application specific integrated circuit (“ASIC”),a programmable gate array (“PGA”), a field programmable gate array(“FPGA”), etc.), or any combination of hardware and software.

At 901, the system receives input of a device sensor signal at time T1,and at 903 the system receives input of a gesture signal at time T2.Time T1 and time T2 may occur simultaneously or non-simultaneously witheach other and in any order. Multiple additional gesture inputs ordevice sensor inputs may be used to give greater precision to thedynamic haptic effect or to provide the dynamic haptic effect over agreater period of time. The gesture signals and the device sensorsignals may be received in any order or time sequence, eithersequentially with non-overlapping time periods or in parallel withoverlapping or concurrent time periods. At 905, the device sensor signalis compared to a haptic effect signal to generate a device sensordifference vector. At 907, the gesture signal is compared to a hapticeffect signal to generate a gesture difference vector. At 909, ananimation or physical model description may optionally be received. At911, an interaction parameter is generated using the gesture differencevector, the signal difference vector, and optionally the animation orphysical model description. It will be recognized that any type of inputsynthesis method may be used to generate the interaction parameter fromone or more gesture signals or device sensor signals including, but notlimited to, the method of synthesis examples listed in TABLE 2 below. At913, a drive signal is applied to a haptic actuator according to theinteraction parameter.

TABLE 2 METHODS OF SYNTHESIS Additive synthesis - combining inputs,typically of varying amplitudes Subtractive synthesis - filtering ofcomplex signals or multiple signal inputs Frequency modulationsynthesis - modulating a carrier wave signal with one or more operatorsSampling - using recorded inputs as input sources subject tomodification Composite synthesis - using artificial and sampled inputsto establish a resultant “new” input Phase distortion - altering thespeed of waveforms stored in wavetables during playback Waveshaping -intentional distortion of a signal to produce a modified resultResynthesis - modification of digitally sampled inputs before playbackGranular synthesis - combining of several small input segments into anew input Linear predictive coding - similar technique as used forspeech synthesis Direct digital synthesis - computer modification ofgenerated waveforms Wave sequencing - linear combinations of severalsmall segments to create a new input Vector synthesis - technique forfading between any number of different input sources Physical modeling -mathematical equations of the physical characteristics of virtual motion

FIG. 10 is a flow diagram for concurrently applying a haptic effect tomultiple devices according to one embodiment of the present invention.At 1001, the system enables a unidirectional or bidirectionalcommunication link between a first device having a first haptic actuatorand a second device having a second haptic actuator. At 1003, the systemreceives input of a first gesture signal or device sensor signal fromthe first device and communicates it to the second device via thecommunication link. At 1005, the system optionally receives input of asecond gesture signal or device sensor signal from the second device andcommunicates it to the first device via the communication link. At 1007,an interaction parameter is generated using the first gesture or devicesensor signal and the optional second gesture or device sensor signal.At 1009, a drive signal is concurrently applied to the haptic actuatoron the first device and the second haptic actuator on the second deviceaccording to the interaction parameter. In one embodiment, theinteraction parameter is generated independently on each device. Inanother embodiment, the interaction parameter is generated once on onedevice and then communicated to the other device via the communicationlink.

FIG. 11 is a flow diagram for encoding and applying a haptic effectusing a data file according to one embodiment of the present invention.At 1101, the system receives input of a gesture signal or device sensorsignal from a first device. At 1103, the gesture or device sensor signalis stored or encoded into a data file on the first device. At 1105, thedata file is communicated to a second device having a haptic actuatorvia any file transfer mechanism or communication link. At 1107, thesecond device reads the stored gesture or device sensor signal from thedata file on the second device. At 1109, a drive signal is applied tothe haptic actuator on the second device according to the gesture ordevice sensor signal.

Several embodiments are specifically illustrated and/or describedherein. However, it will be appreciated that modifications andvariations of the disclosed embodiments are covered by the aboveteachings and within the purview of the appended claims withoutdeparting from the spirit and intended scope of the invention.

1-30. (canceled)
 31. A method of producing a haptic effect comprising:enabling a communication link between a first device having a firsthaptic actuator and a second device having a second haptic actuator;receiving a first signal from the first device and communicating it tothe second device via the communication link; generating an interactionparameter using the first signal; and concurrently applying a drivesignal to the first actuator and the second actuator according to theinteraction parameter.
 32. The method of claim 31 further comprising:receiving a second signal from the second device and communicating it tothe first device via the communication link; and wherein generating aninteraction parameter comprises generating an interaction parameterusing the first signal and the second signal.
 33. The method of claim 31wherein the first signal comprises a vector signal.
 34. The method ofclaim 31 wherein the first signal comprises an on-screen signal.
 35. Themethod of claim 32, wherein generating an interaction parametercomprises generating an interaction parameter from a combination of thefirst signal and the second signal.
 36. The method of claim 31, whereinthe communication link comprises a link selected from the listconsisting of electronic, cellular, wireless, wi-fi, optical, infrared,acoustic, Bluetooth, USB, Firewire, Thunderbolt or Ethernet.
 37. Themethod of claim 31 wherein the first signal comprises a gesture signal.38. The method of claim 31 wherein the first signal comprises anaccelerometer signal.
 39. The method of claim 31 wherein the firstsignal comprises a gyroscope signal.
 40. The method of claim 31 whereinthe first signal comprises an ambient signal.
 41. The method of claim 31wherein the first signal comprises a virtual sensor signal.
 42. A methodof producing a haptic effect comprising: receiving a signal from a firstdevice; encoding the signal in a data file on the first device;communicating the data file to a second device having a haptic actuator;reading the signal from the data file on the second device; and applyinga drive signal to the actuator according to the signal.
 43. The methodof claim 42 wherein the signal comprises a vector signal.
 44. The methodof claim 42 wherein the signal comprises an on-screen signal.
 45. Themethod of claim 42 wherein the signal comprises a gesture signal. 46.The method of claim 42 wherein the signal comprises an accelerometersignal.
 47. The method of claim 42 wherein the signal comprises agyroscope signal.
 48. The method of claim 42 wherein the signalcomprises an ambient signal.
 49. The method of claim 42 wherein thesignal comprises a virtual sensor signal.
 50. The method of claim 42wherein encoding the signal comprises encoding an interaction parameteraccording to the signal, and wherein reading the signal comprisesreading the interaction parameter from the data file on the seconddevice, and wherein applying a drive signal comprises applying a drivesignal to the actuator according to the interaction parameter.